Image processing method and device

ABSTRACT

By the use of an epsilon filter, in the case where a plurality of different illuminations exist, a boundary between the illuminations can be appropriately extracted, and an unnatural image pattern can be prevented from being generated, therefore subjectively preferable compression of a dynamic range can be achieved. An edge strength G(x, y) is calculated per position on an input image, and a threshold E(x, y) of an epsilon filter ( 12 ) is controlled on the basis of the edge strength G(x, y). The epsilon filter ( 12 ) filters the input image on the basis of the controlled threshold E(x, y). On the basis of the edge strength G(x, y), the threshold E of the epsilon filter ( 12 ) is adaptively changed according to a local gradient of a pixel value I(x, y), so in the case of using a linear lowpass filter or a fixed threshold epsilon filter, an illumination boundary can be more accurately extracted.

TECHNICAL FIELD

[0001] The present invention relates to an image processing method andan image processing apparatus suitably applicable to various imageinput/output devices such as televisions, video tape recorders, stillcameras, video cameras and printers, and more specifically to an imageprocessing method and an image processing apparatus for reproducing aninputted image in an imaging device with a relatively narrow dynamicrange.

BACKGROUND ART

[0002] As a conventional method, there is a method (hereinafter referredto as “level conversion”) of converting each pixel of an input image bya function having an input/output relationship (hereinafter referred toas “level conversion function”) indicated by a solid line in FIG. 18,for example, for conversion of gradation characteristics of an image. InFIG. 18, a lateral axis indicates a pixel level (input level) l of aninput image, and a vertical axis indicates a pixel level (output level)T(l) of an output image by the level conversion. Lmax indicates amaximum level which each pixel of an input/output image can obtain. Acontrast of the image after the level conversion increases with anincrease in a gradient of the level conversion function. In an exampleshown in FIG. 18, gradients of straight lines indicating the levelconversion function in a high level from an input level lb as a borderof the high level and in a low level from an input level ls as a borderof the low level are smaller than a gradient in a medium level (from theinput level ls to the input level lb). Therefore, in the levelconversion using the function shown in FIG. 18, a contrast in the mediumlevel is increased by sacrificing contrasts in the high level and thelow level.

[0003] In addition to the level conversion function shown in FIG. 18, alevel conversion function indicated by a solid line in FIG. 19 can beused. In the level conversion function shown in FIG. 19, a gradient of astraight line in the high level from an input level lk as a boundary ofthe high level is smaller than gradients in the low level and the mediumlevel. Therefore, in the level conversion using the function shown inFIG. 19, the contrasts in the low level and the medium level can beincreased by sacrificing the contrast in the high level. Further,compared with the functions shown in FIGS. 18 and 19, a more continuouslevel conversion function such as a gamma function shown in MathematicalFormula 1 or a LOG function shown in Mathematical Formula 2 may be used.In Mathematical Formula 1, “g” indicates a parameter for adjusting thegradient of the function.

[0004] Moreover, as another conventional method, there is a method ofadaptively changing the level conversion function according to afrequency distribution of the pixel level of the input image, and as atypical example of the conventional method, a method called histogramequalization is cited. FIGS. 20A and 20B show a principle of thehistogram equalization. In FIG. 20A, a lateral axis indicates a pixellevel (input level) l of an input image, and a vertical axis indicatesfrequency (or cumulative frequency). Fmax indicates a maximum value ofthe cumulative frequency, which is a total number of pixels used forcalculating the frequency. In the method, as shown in FIG. 20A, at firsta frequency distribution H(l) relating to the pixel level l of the inputimage is produced, and then a cumulative frequency distribution C(l) isproduced by the use of Mathematical Formula 3.

[0005] The vertical axis of the cumulative frequency distribution C(l)is normalized to a level range in which the output image can obtain bythe use of Mathematical Formula 4 so as to produce a level conversionfunction T(l) (refer to FIG. 20B). By the use of the function T(l), acontrast in a region configured with a high frequency level (a regionwith a large area) can be increased.

[0006] When an inputted image is used in an environment where thedynamic range is smaller, that is, the number of bits representing thepixel level is smaller (for example, in the case of transmitting theimage through a transmission line with a small number of bits,displaying the image on a display apparatus, or storing the image inmemory), the dynamic range is required to be compressed. Conventionally,the same level conversion as the method described above is used tocompress the dynamic range for such purpose. However, in this case, amaximum level of the output image of the level conversion function has asmaller value than that of the input image.

[0007] On the other hand, in literature of “Z. Rahman, et, alt.:“AMultiscale retinex for color rendition and dynamic range compression inApplications of Digital image Processing”, XIX Proc. SPIE 2847 (1996)”,a method (hereinafter referred to as “Multiscale retinex method”) ofcompressing the entire dynamic range by extracting and compressing acomponent of illumination light which is spatially and slightly changedby the use of a lowpass filter is proposed. A linear narrow-band lowpassfilter is used to extract an illumination component. In the method, asshown in Mathematical Formula 5, a logarithm value of an input pixelvalue I(x, y) and a logarithm value of a lowpass filter output LPF(I(x,y)) are taken, and then the latter is subtracted from the former tocompress the dynamic range.

[0008] In the above conventional level conversion methods, in order toprevent from producing an unnatural image, a level conversion functionhaving a monotone increasing property is used. Therefore, there is aproblem that when a contrast in any level range (the gradient of thelevel conversion function) is increased, conversely, contrasts in otherlevel ranges declines.

[0009] Further, in the Multiscale retinex method, by sacrificing themonotone increasing property, an image with a higher contrast can bereproduced. However, there is a problem that when an illuminationcondition is suddenly switched, the linear filter cannot extract achange in the illumination condition, so a subjectively undesirablenoise occurs.

[0010] For example, as shown in FIG. 21, when an image having tworegions with different illumination conditions adjacent to each other(indicated by a solid line in the drawing) is filtered by the linearlowpass filter, a signal with an ambiguous boundary indicated by a thinbroken line is obtained as a filter output. When the filter output isconsidered as the illumination component, in a region on a left side ofan illumination boundary (B region), a portion near the boundary (BNBregion) has a lower illumination level than a portion at a distance fromthe boundary (BFB region). The Mathematical Formula 5 is equivalent todividing an input signal by the illumination component, and means thatthe larger the illumination component is, the more the input signal iscompressed. Accordingly, overshoot occurs in the BNB region of areproduced image (indicated by a thick broken line in the drawing).Conversely, it is considered that in a region on a right side of theillumination boundary (D region), a portion near the boundary (DNBregion) has a higher illumination level than a portion at a distancefrom the boundary (DFB region), so undershoot occurs. In the Multiscaleretinex method, in order to overcome the problem, a method of using aplurality of linear lowpass filters with different scales, andsynthesizing results obtained by each of the lowpass filters by a linearload is used, but a weight for each scale is fixed, so the above problemcannot be sufficiently prevented.

[0011] Therefore, it is considered that a nonlinear filter such as, forexample, an epsilon filter instead of the linear lowpass filter is usedto extract the illumination component. The epsilon filter is superior inperformance of storing an edge to the linear filter, so the illuminationcomponent can be effectively extracted from an image where differentillumination lights exist. However, in a fixed threshold epsilon filterwhich is generally used to remove noise, a discontinuous waveform isgenerated in an output thereof in a neighborhood of the edge, so whenthe filter is used to compress the dynamic range, an unnatural imagepattern which does not exist in an original image may be generated in areproduced image after compression.

[0012] In view of the foregoing, it is an object of the invention toprovide an image processing method and an image processing apparatuscapable of appropriately extracting a boundary between a plurality ofilluminations by the use of the epsilon filter in the case where theplurality of illuminations exist, and preventing from generating anunnatural image pattern so as to achieve subjectively preferablecompression of the dynamic range.

DISCLOSURE OF THE INVENTION

[0013] An image processing method according to the invention comprisesthe steps of: calculating an edge strength per position on an inputimage; controlling a threshold of an epsilon filter on the basis of thecalculated edge strength; filtering the input image by the epsilonfilter by use of the threshold controlled in the step of controlling thethreshold; and calculating a coefficient for converting a pixel valueaccording to an output value in the step of filtering, and convertingthe pixel value per pixel by the calculated coefficient.

[0014] An image processing apparatus according to the inventioncomprises: an edge strength calculating means for calculating an edgestrength per position on an input image; an epsilon filter for filteringthe input image by use of a predetermined threshold; a threshold controlmeans for controlling the threshold used in the epsilon filter on thebasis of the edge strength calculated by the edge strength calculatingmeans; and a pixel value conversion means for calculating a coefficientfor converting a pixel value according to an output value from theepsilon filter and converting the pixel value per pixel by thecalculated coefficient.

[0015] In the image processing method and the image processing apparatusaccording to the invention, the edge strength is calculated per positionon the input image, and the threshold used in the epsilon filter iscontrolled on the basis of the calculated edge strength. Then, acoefficient for converting a pixel value is calculated according to anoutput value from the epsilon filter, and the pixel value is convertedper pixel by the calculated coefficient. Thereby, in the case where aplurality of different illuminations exit, in filtering by the epsilonfilter, a boundary between the illuminations can be appropriatelyextracted.

[0016] In the invention, for example, the threshold of the epsilonfilter is controlled so that the larger the edge strength is, thesmaller a value of the threshold becomes. In this case, the threshold ofthe epsilon filter may be further controlled so that the larger a pixelvalue of the input image is, the larger the threshold becomes. Thereby,in filtering by the epsilon filter, an influence of an illuminationlevel on a change in the pixel value can be reduced, so an illuminationcomponent can be appropriately extracted.

[0017] Moreover, two thresholds with different values may be calculatedso as to control the threshold. In this case, in the epsilon filter, forexample, filtering is performed by the use of a different thresholddepending upon whether a value of a neighboring pixel is larger orsmaller than a value of a pixel of interest. Thereby, in filtering, aninfluence of the illumination level on a change in the pixel value canbe reduced, so the illumination component can be more appropriatelyextracted.

BRIEF DESCRIPTION OF THE DRAWINGS

[0018]FIG. 1 is a block diagram showing a structure of an imageprocessing apparatus according to a first embodiment of the invention;

[0019]FIG. 2 is an illustration for explaining a scanning direction ofan image;

[0020]FIG. 3 is an illustration for explaining an example of a levelconversion function;

[0021]FIG. 4 is a block diagram showing a structure of an epsilon filterin the image processing apparatus shown in FIG. 1;

[0022]FIGS. 5A through 5C are illustrations for explaining a differencebetween a reproduced image by a conventional method and a reproducedimage by the image processing apparatus shown in FIG. 1;

[0023]FIGS. 6A and 6B are illustrations for explaining a relationshipbetween the level conversion function and a coefficient calculationfunction;

[0024]FIGS. 7A and 7B are illustrations for explaining an effect of theepsilon filter;

[0025]FIGS. 8A and 8B are illustrations for explaining behavior of afixed threshold epsilon filter in a neighborhood of an edge;

[0026]FIGS. 9A and 9B are illustrations for explaining an inversionphenomenon of a level gradient which occurs in the fixed thresholdepsilon filter;

[0027]FIG. 10 is an illustration for explaining a relationship betweenthe level conversion function used for an output of the epsilon filterand a differential value thereof;

[0028]FIGS. 11A and 11B are illustrations for explaining a inequalityrelationship between an input level and a compression ratio in a levelconversion curve;

[0029]FIG. 12 is a block diagram showing a structure of an imageprocessing apparatus according to a second embodiment of the invention;

[0030]FIG. 13 is an illustration for explaining an example of acoefficient calculation function used in a coefficient calculator in theimage processing apparatus shown in FIG. 12;

[0031]FIG. 14 is a block diagram showing a structure of an imageprocessing apparatus according to a third embodiment of the invention;

[0032]FIG. 15 is a block diagram showing a structure of an imageprocessing apparatus according to a fourth embodiment of the invention;

[0033]FIG. 16 is an illustration for explaining a structure of anepsilon filter in the image processing apparatus shown in FIG. 15;

[0034]FIG. 17 is a block diagram showing a structure of an imageprocessing apparatus according to a fifth embodiment of the invention;

[0035]FIG. 18 is an illustration for explaining an example of aconventionally used level conversion function;

[0036]FIG. 19 is an illustration for explaining an example of anotherconventionally used level conversion function;

[0037]FIGS. 20A and 20B are illustrations for explaining a principle ofhistogram equalization; and

[0038]FIG. 21 is an illustration for explaining a problem of Multiscaleretinex method.

BEST MODE FOR CARRYING OUT THE INVENTION

[0039] Preferred embodiments of the present invention will be describedin more detail below referring to the accompanying drawings.

[0040] [First Embodiment]

[0041] At first, an input image signal which is processed in an imageprocessing apparatus according to a first embodiment of the inventionwill be described below. The input image signal processed in the imageprocessing apparatus is a signal of a time-series pixel value obtainedby scanning a two-dimensional digital image in a horizontal directionand a vertical direction in this order as shown in FIG. 2. In theembodiment, the pixel value corresponding to any position (x, y) on thetwo-dimensional image indicates I(x, y), and the pixel value isprocessed as an input image signal.

[0042] Next, a structure of the image processing apparatus according tothe embodiment will be described below. As shown in FIG. 1, the imageprocessing apparatus comprises an edge strength calculator 10, athreshold controller 11, an epsilon filter 12, a divider 13, a levelconversion device 14 and a multiplier 15.

[0043] The edge strength calculator 10 has a function of calculating anedge strength G(x, y) of the pixel value I(x, y) in each position on theinput image. As the edge strength G(x, y), a primary differential valueof I(x, y) given by, for example, Mathematical Formula 6 can be used.

[0044] Alternatively, a value by Mathematical Formula 7 having asmoothing effect for reducing an influence of noise can be used as theedge strength G(x, y).

[0045] In Mathematical Formulas 6 and 7, “d” is a constant indicating aninfinitesimal distance for calculating a differential. The edge strengthG(x, y) calculated by the edge strength calculator 10 is transmitted tothe threshold controller 11.

[0046] The threshold controller 11 has a function of determining themagnitude of a threshold E(x, y), which is used in the epsilon filter 12to be described later, per pixel on the basis of the edge strength G(x,y) calculated by the edge strength calculator 10. By the function of thethreshold controller 11, the threshold E(x, y) is controlled by the useof, for example, Mathematical Formula 8 so that the larger the edgestrength G(x, y) is, the smaller value the threshold E(x, y) becomes.

[0047] In Mathematical Formula 8, Gmin, Gmax, Emin and Emax areconstants for converting the edge strength G(x, y) into the thresholdE(x, y), and indicate a minimum value of the edge strength, a maximumvalue of the edge strength, a minimum value of the threshold E(x, y) anda maximum value of the threshold E(x, y), respectively. The thresholdE(x, y) determined by the threshold controller 11 is transmitted to theepsilon filter 12.

[0048] As shown in FIG. 4, the epsilon filter 12 includes, for example,a difference device 20, an absolute value calculator 21, a comparator 22and a linear lowpass filter (LPF in the drawing) 23. The epsilon filter12 is a two-dimensional filter, and has a function of nonlinearlyfiltering the input image by the use of the threshold E(x, y) determinedby the threshold controller 11. An output R(x, y) of the epsilon filter12 is transmitted to the divider 13 and the level conversion device 14as an illumination component.

[0049] In order to remove the illumination component calculated by theepsilon filter 12 from the input image, as shown in Mathematical Formula9, the divider 13 performs a division of each pixel value I(x, y) of theinput image by the illumination component R(x, y). A non-illuminationcomponent S(x, y) obtained as a result of the division is transmitted tothe multiplier 15.

[0050] The level conversion device 14 has a function of performing levelconversion on the illumination component R(x, y) calculated by theepsilon filter 12 by a level conversion function T(l) as shown inMathematical Formula 10 to compress the illumination component R(x, y),and thereby calculating a correction illumination component CR(x, y).

[0051] As the level conversion function T(l) used in the levelconversion device 14, for example, a function shown in FIG. 3 can beused. In FIG. 3, Rmax and CRmax indicate maximum values of an inputlevel and an output level, respectively.

[0052] The multiplier 15 multiplies the non-illumination component S(x,y) by the correction illumination component CR(x, y) as shown inMathematical Formula 11 to reproduce the image signal. An image signalO(x, y) indicating a reproduced result is outputted to a transmissionline, memory, a display apparatus or the like (all not shown).

[0053] In the embodiment, the divider 13, the level conversion device 14and the multiplier 15 correspond to a specific example of “a pixel valueconversion means” in the invention.

[0054] Next, effects and actions of the image processing apparatus withthe above structure will be described below. The description belowincludes a description of an image processing method according to theembodiment.

[0055] In the image processing apparatus, a signal indicating the inputimage is inputted into the edge strength calculator 10, the epsilonfilter 12 and the multiplier 13. At first, in the edge strengthcalculator 10, the magnitude of the edge, that is, the edge strengthG(x, y) per position on the input image is calculated. At this time, theedge strength calculator 10 calculates the edge strength G(x, Y) by theuse of, for example, Mathematical Formula 6 or 7 so that the larger aprimary differential value of a pixel value in a neighboring region of apixel of interest is, the larger the edge strength G(x, y) becomes. Theedge strength calculator 10 outputs the calculated edge strength G(x, y)to the threshold controller 11.

[0056] The threshold controller 11 controls a threshold E of the epsilonfilter 12 on the basis of the edge strength G(x, y). More specifically,the threshold controller 11 determines the magnitude of the thresholdE(x, y) per pixel by the use of, for example, Mathematical Formula 8 tocontrol the threshold E(x, y) so that the larger the edge strength G(x,y) is, the smaller the threshold E(x, y) becomes. The thresholdcontroller 11 outputs the determined threshold E(x, y) to the epsilonfilter 12.

[0057] The epsilon filter 12 filters the input image by the use of thethreshold E(x, y) determined by the threshold controller 11.

[0058] More specifically, filtering in the epsilon filter 12 isperformed by, for example, the structure shown in FIG. 4 as follows. Inthe epsilon filter 12, as shown in FIG. 4, a signal indicating a valueI(x, y) of a pixel of current interest and a signal indicating a valueI(x+dx, y+dy) of a pixel in a neighboring region NB of the pixel ofcurrent interest are inputted into the difference device 20. Thedifference device 20 calculates a difference between the value I(x, y)of the pixel of interest and the value I(x+dx, y+dy) of the pixel in theneighboring region NB. The difference device 20 successively calculatesdifference values between the value I(x, y) of the pixel of interest andvalues of all pixels in the neighboring region NB, and assigns adifference value D(dx, dy) to each neighboring pixel to output the valueD(dx, dy) to the absolute value calculator 21.

[0059] The absolute value calculator 21 calculates an absolute valueAD(dx, dy) of each difference value D(dx, dy) transmitted from thedifference device 20. The absolute value calculator 21 outputs theabsolute value AD(dx, dy) to the comparator 22.

[0060] The absolute value AD(dx, dy) calculated in the absolute valuecalculator 21 as well as the signal indicating the value I(x, y) of thepixel of interest, the signal indicating the value I(x+dx, y+dy) of thepixel in the neighboring region NB and the threshold E(x, y) determinedby the threshold controller 11 are inputted into the comparator 22. Thecomparator 22 performs a comparison between the absolute value AD(dx,dy) and the threshold E(x, y) as shown in Mathematical Formula 12, andselects either the value I(x, y) of the pixel of interest or the valueI(x+dx, y+dy) of the neighboring pixel according to the result of thecomparison to output the selected value as a value J(dx, dy) to thelinear lowpass filter 23.

[0061] The linear lowpass filter 23 calculates a weighted average valueR(x, y) by Mathematical Formula 13 when the value J(dx, dy)corresponding to all pixels in the neighboring region NB is calculatedby the comparator 22.

[0062] Herein, NB indicates a set of relative coordinates defining aneighboring region in filtering. Further, a(dx, dy) indicates aweighting coefficient for each pixel value. As the linear lowpass filter23, an average filter shown in, for example, Mathematical Formula 14 canbe used.

[0063] In Mathematical Formula 14, N indicates a number of pixels in theneighboring region NB. The purpose of the epsilon filter 12 is to removea fine structure in the image and extracts a massed region, so a largeneighboring region is preferable.

[0064] As described above, the value R(x, y) obtained in the epsilonfilter 12 is considered as a value approximately indicating theillumination component included in the image. The epsilon filter 12outputs the value R(x, y) as the illumination component to the divider13 and the level conversion device 14.

[0065] The divider 13 divides each pixel value I(x, y) of the inputimage by the illumination component R(x, y) as shown in MathematicalFormula 9 so as to remove the illumination component calculated by theepsilon filter 12 from the input image and output a non-illuminationcomponent S(x, y) obtained as a result to the multiplier 15.

[0066] On the other hand, the level conversion device 14 compresses theillumination component R(x, y) calculated by the epsilon filter 12through level conversion with, for example, the level conversionfunction T(l) shown in FIG. 3 so as to calculate a correctionillumination component CR(x, y). The level conversion device 14 outputsthe calculated correction illumination component CR(x, y) to themultiplier 15.

[0067] The multiplier 15 multiples the non-illumination component S(x,y) outputted from the divider 13 by the correction illuminationcomponent CR(x, y) outputted from the level conversion device 14 toreproduce the image signal. Herein, when the whole computation by thedivider 13, the level conversion device 14 and the multiplier 15 isconsidered, as shown in Mathematical Formula 16, a multiplication of thenon-illumination component S(x, y) by the correction illuminationcomponent CR(x, y) corresponds to calculating a coefficient F(R(x, y))for converting the pixel value according to the output value R(x, y)from the epsilon filter 12, and then multiplying the coefficient F(R(x,y)) by the corresponding input pixel value I(x, y) so as to compress thedynamic range by conversion of the pixel value per pixel.

[0068] As described above, the image signal O(x, y) outputted from themultiplier 15 is used in an imaging device with relatively narrowerdynamic range than the input image, that is, in an environment of asmall number of bits indicating the pixel level (in the case oftransmitting through a transmission line with a small number of bits,the case of displaying on a display device or the case of storing inmemory).

[0069] Next, referring to FIGS. 5A through 6B, effectiveness of the casewhere the dynamic range is compressed by the epsilon filter 12 accordingto the embodiment against a conventional method (the case wherecompression is performed by the use of the level conversion functionwith a monotone increasing property) will be described below.

[0070]FIG. 5A is an illustration showing the pixel value I(x, y) of theinput image and the output R(x, y) from the epsilon filter 12 accordingto the embodiment as one-dimensional signals. FIG. 5B shows a result (areproduced image) of compressing the dynamic range of the input imageshown in FIG. 5A by a conventional level conversion method, and FIG. 5Cshows a result of compressing the dynamic range according to theembodiment.

[0071] Moreover, FIGS. 6A and 6B show a relationship among a pixel levelin each region, the level conversion function T(l) and a coefficientcalculation function F(l). Herein, the coefficient calculation functionF(l) is defined as Mathematical Formula 15 by the use of the levelconversion function T(l).

[0072] By consideration of Mathematical Formulas 9 and 10 using thecoefficient calculation function F(l), Mathematical Formula 11 givingthe output image O(x, y) can be rewritten to Mathematical Formula 16.

[0073] The Mathematical Formula 16 shows that the compression of thedynamic range by compression of the illumination component R(x, y) canbe achieved by multiplying the coefficient F(R(x, y)) calculated perpixel by the corresponding input pixel value I(x, y). In this case, thecoefficient calculation function F(l) has a function of converting theoutput value of the epsilon filter 12 into a gain coefficient applied toeach pixel. Further, a minimum value Cmin of the coefficient calculationfunction F(l) in FIG. 6B is given by Mathematical Formula 17.

[0074] As can be seen from FIG. 5B, in the conventional method, acontrast in a low level region (a region consisting of a level l1 and alevel l3 shown in FIG. 5A) can be stored, but a contrast in a high levelregion (a region consisting of a level l4 and a level l6) declines. Itis a result of directly receiving an influence of the gradient of thelevel conversion function T(l) on a high level of an inflection point 1kor over. In the conventional method, in order to improve the contrast,it is required to increase the gradient of the level conversion functionT(l).

[0075] On the other hand, in the embodiment (refer to FIG. 5C), a singlecorrection coefficient given by the coefficient calculation functionF(l) is applied to each of the high level region and the low levelregion, so the contrast in each region depends upon the magnitude of thecorrection coefficient. In the embodiment, the correction coefficientdetermined by an average level l2 is uniformly applied to the low levelregion, but the value of the correction coefficient is the same 1.0 asthat applied to the level l1 and the level l3, so the same level of thecontrast as that in the conventional method can be obtained. Moreover, afixed correction coefficient c5 determined by an average value l5 isapplied to the high level region, so the contrast between a portion ofthe level l4 and a portion of the level l6 can be improved.

[0076] In reality, when a direct line corresponding to a level of theinflection point 1k or over is indicated by Mathematical Formula 18 bythe use of the level conversion function T(l) consisting of two directlines with different gradients as shown in FIG. 6A, the contrast in thehigh level region in the conventional method depends upon a gradient aof the direct line, and is given by“a(l6−l4)/(a*l5+b)=(l6−l4)/(l5+b/a)”. Herein, the contrast is defined by(maximum level minimum level)/average level.

[0077] On the other hand, the correction coefficient c5 applied to thehigh level region in the embodiment is given by Mathematical Formula 19.

[0078] Therefore, in the embodiment, the contract in the region isc5(l6−l4)/c5*l5=(l6−l4)/l5, but the gradient of the level conversionfunction T(l) in compression of the dynamic range in the high level isgenerally less than 1.0, so an intercept b is always a positive value.This indicates that compared with the conventional method, a methodusing the epsilon filter 12 according to the embodiment can achieve ahigher contrast.

[0079] Thus, in the embodiment, a contrast in a region extracted by theepsilon filter 12 is determined by the value of the correctioncoefficient given by the coefficient calculation function F(l), so thegradient of the level conversion function T(l) has an influence on acontract between regions. Therefore, according to the embodiment,compression of the contract between the regions allows storing thecontrast in the region, thereby subjectively preferable output image canbe obtained.

[0080] Next, effects of the embodiment on a conventional method usingthe linear lowpass filter will be described below. Herein, for the sakeof simplifying the description, the image is indicated as aone-dimensional signal. The problems of the conventional method havebeen already described in paragraphs of “Background Art” referring toFIG. 21, and in order to overcome the problems, while a boundary(illumination boundary) between regions with different illuminationconditions is stored, it is required to smooth a region under the sameillumination. Experientially, a change in the pixel level resulting froma change in an illumination strength is much larger than a change in thepixel level resulting from a reflectivity of a surface of an object.Consequently, an edge with a large pixel level occurs in theillumination boundary.

[0081] As shown in FIG. 7A, in proximity to such edge, the pixel valueI(x+dx, y+dy) giving a large difference absolute value AD(dx, dy)exceeding the threshold E(x, y) exists in the neighboring region NB ofthe pixel of interest in the epsilon filter 12. The pixel value I(x+dx,y+dy) can be replaced with the value I(x, y) of the pixel of currentinterest (a value at the center of the neighboring region NB) asindicated by a thick broken line (I′(dx, dy)) in FIG. 7A by thecomparator 22 (refer to FIG. 4), so the pixel value I(x+dx, y+dy) doesnot greatly contribute smoothing by Mathematical Formula 13, andconsequently, the shape of the edge is stored. On the other hand, in aportion except for the illumination boundary, as shown in FIG. 7B, achange in the pixel level is not so large and the difference absolutevalue AD(dx, dy) is smaller than the threshold E(x, y) in the wholeneighboring region NB of the pixel of interest. In this case, all of thevalue J(dx, dy) in Mathematical Formula 12 is equal to the input pixelvalue I(x+dx, y+dy), and the epsilon filter is equivalent to a simplelinear lowpass filter, so the whole neighboring region NB is smoothed.

[0082] Thus, the epsilon filter 12 is superior in performance of storingthe edge to the linear filter, and can effectively extract theillumination component from an image with different illumination lights.However, in a conventional fixed threshold epsilon filter generally usedto remove noise, a discontinuous waveform is generated in the outputthereof in proximity to the edge, so when the epsilon filter is used tocompress the dynamic range, an unnatural image pattern which does notexist in an original image may be generated in the reproduced imageafter compression.

[0083] For the sake of describing the problem, FIGS. 8A and 8B show theoutput of the conventional fixed threshold epsilon filter in a modelededge and its proximity. Herein, only a portion from an edge centerportion to the high level side is taken into consideration, and an edgeportion where the level is steeply changed approximates to a direct lineof a gradient a, and a portion in proximity to the edge except for theedge portion approximates to a flat direct line of a gradient 0. InFIGS. 8A and 8B, the input signal, the output of the epsilon filter andthe output of the corresponding linear lowpass filter are indicated by athin solid line 81, a thick solid line 82 and a thick broken line 83,respectively. The linear lowpass filter used in the epsilon filter is anaverage filter. Further, the size of the neighboring region of the pixelof interest in the used epsilon filter is N, and the threshold thereofis E. FIG. 8A shows the output of the epsilon filter when therelationship of Mathematical Formula 20 among a, N, and E isestablished.

[0084] In FIG. 8A, a lateral axis indicates a spatial positioncoordinate, and the value of the coordinate gradually increases towardthe right. “p0” indicates a position at a distance of N/2 from a fallingpoint pe of the edge. In this case, the epsilon filter behaves asequivalent to the linear lowpass filter. However, a level change in theimage due to a change in illumination light steeply occurs (the gradienta is large), and a large filter is required to be used (N is large) inorder to effectively compress the dynamic range as described above, soit is considered that Mathematical Formula 20 is not generallyestablished in the illumination boundary where an illumination conditionis changed.

[0085] On the other hand, FIG. 8B shows the output of the epsilon filterwhen Mathematical Formula 20 is not established. In FIG. 8B, in thewaveform of the output from the right to the left, the output from p0 top0-E/a is the same as that of the linear filter, and the output fromp0-E/a to pe is a uniform value. The output from pe to pe-E/a declinesaccording to a quadratic curve, however, under the condition whereMathematical Formula 20 is not established, the output never fails tointersect with the direct line 81 indicating the input signaltherebetween. The output in pe-E/a becomes discontinuously the samevalue as that of the input signal, and then the input signal isoutputted as it is. This behavior of the fixed threshold epsilon filteris for a simplified edge model, but it is obvious that a complicatedwaveform is outputted in proximity to the edge. Specifically, aroundpe-E/a, a waveform which is discontinuously changed, and has a largergradient than the input signal is outputted.

[0086] In order to make the output image O(x, y) a natural reproducedimage, a local shape (a direction of a spatial gradient) in eachposition on the image is required to be stored. In other words, as shownin Mathematical Formula 21, a sign of a differential value O′(x, y) ofthe output image O(x, y) and a sign of a differential value I′(x, y) ofthe input image I(x, y) must match each other.

[0087] Herein, a sign (x) indicates a sign of x. When the condition ofMathematical Formula 21 is not satisfied, inversion of a level gradientbetween the input image and the output image occurs, and thereby animage pattern which does not exist in the input image is generated inthe output image. For example, the input image shown in FIG. 9A isoutputted as shown in FIG. 9B. In the output image in FIG. 9B, a pattern90 which does not exist in the input image is generated due to theinversion of the level gradient. However, a discontinuous behavior ofthe epsilon filter in proximity to the edge shown in FIG. 8B may causedifficulty in establishing the condition of Mathematical Formula 21,which thereby may result in the occurrence of inversion of the levelgradient.

[0088] Whether or not establishing Mathematical Formula 21 depends uponthe behavior of the epsilon filter and the level conversion functionT(l) applied to the output. In order to prove it, Mathematical Formula 9is substituted into Mathematical Formula 11 to obtain MathematicalFormula 22 indicating a relationship of the input image I(x, y) and theoutput image O(x, y).

[0089] CR(x, y)/R(x, y) in the right side of Mathematical Formula 22corresponds to the above-described coefficient calculation functionF(R(x, y)). The both sides of Mathematical Formula 22 are differentiatedto substitute into Mathematical Formula 21, thereby the condition whichthe output image O(x, y) must satisfy indicates like MathematicalFormula 23.

[0090] In order to make clearer the involvement of the level conversionfunction T(l) in the condition, FIG. 10 shows a relationship of thelevel conversion function T(l) to the output of the epsilon filter. Alateral axis indicates the output of the epsilon filter (input into thelevel conversion function T(l)), and a vertical axis indicates theoutput of the level conversion function T(l). Herein, an output of theepsilon filter of an pixel is R, a value of the output after the levelconversion is CR, and a value of their ratio CR/R is a. In other words,“a” corresponds to a gradient of a direct line passing through an originpoint and (R, CR) (Mathematical Formula 24).

[0091] Further, where a spatial differential value of the output of theepsilon filter in the same pixel is R′, R′ is considered to correspondto a size of a minute range with R on the lateral axis in FIG. 10 as acenter (R has an adequate sign according to a direction of calculatingthe differential value on the image). Therefore, where a differentialcoefficient regarding “l” of the level conversion function T(l)corresponding to R is b, a spatial differential value after the levelconversion can approximate by Mathematical Formula 25.

[0092] By substituting Mathematical Formulas 24 and 25 into MathematicalFormula 22, a condition of Mathematical Formula 26 consisting of aportion R′/R regarding the output of the epsilon filter and a portion(1-b/a) regarding a property of the level conversion function T(l) canbe obtained.

[0093] Incidentally, the level conversion function T(l) for compressingthe illumination component generally has the monotone increasingproperty, so values of a and b are positive, so Mathematical Formula 27is established.

[0094] Further, 1-b/a∞0 means a∞b, and in this case, it means that in aneighborhood of an illumination component level R, the larger the levelis, the larger the compression is performed. It is because a compressionratio of the level R is a, and when the neighborhood thereof isuniformly compressed, a gradient of the level conversion function T(l)in the neighborhood of the level R is required to be a, however, theactual gradient is b which is smaller, so as shown in FIG. 11A, thehigher the level side is, the larger compression is performed. On theother hand, in the case of 1-b/a, that is, a<b, as shown in FIG. 11B,the lower the level side is, the larger compression is performed. As inthe case of FIG. 10, the vertical axes and the lateral axes of FIGS. 11Aand 11B indicate the output of the epsilon filter (the input into thelevel conversion function T(l)) and the output of the level conversionfunction T(l), respectively.

[0095] In a flat portion except for a portion in proximity to the edge,the epsilon filter functions as a lowpass filter, so a ratio of aspatial change in the output is generally smaller than that in the inputimage. Therefore, it can be assumed that in the portion, two conditionsindicated in Mathematical Formulas 28A and 28B are established.

[0096] In the case of I′∞0, R′∞0 is established by Mathematical Formulas28A and 28B, so if 1-b/a is negative, Mathematical Formula 26 is alwaysestablished. Further, in the case of 0 1-b/a 1, Mathematical Formula 29is established by I′/I∞R′/R obtained by Mathematical Formulas 28A and28B.

[0097] Likewise, in the case of I′<0, it can be easily found thatMathematical Formula 26 is established.

[0098] On the other hand, as shown in FIG. 8B, the epsilon filteroutputs the input signal as it is in the edge center portion, so it isconsidered that Mathematical Formula 30 is established in the portion,and also in this case, it is obvious that Mathematical Formula 26 issatisfied.

[0099] In the embodiment, when the epsilon filter 12 is removed so thatR(x, y)=I(x, y) is established, the output of CR(x, y) is equivalent tothat in the conventional method using only the level conversion, whichcorresponds to the case where the condition of Mathematical Formula 30is established throughout the image, therefore, in the conventionalmethod, inversion of the level gradient will not occur.

[0100] In a portion where these conditions are not satisfied, whether ornot establishing Mathematical Formula 26 depends upon the output of theepsilon filter 12 and the property of the level conversion functionT(l). The property of the level conversion function T(l) should bedetermined by how to compress the illumination component, so adescription will be given of how the output of the epsilon filter 12 hasan influence on the establishment of Mathematical Formula 26 with fourcases (1) through (4).

[0101] (1) In the case where I′∞0 and R′ are the same signs as 1-b/a;

[0102] the values on the right side and the left side are positive, sothe larger the absolute value of R′/R is, the more difficult it is toestablish Mathematical Formula 26.

[0103] (2) In the case where I′∞0 and R′ are different signs from 1-b/a;

[0104] the value on the right side is positive and the value on the leftside is negative, so Mathematical Formula 26 is always established.

[0105] (3) In the case where I′<O and R′ are the same signs as 1-b/a;

[0106] the value on the right side is negative and the value on the leftside is positive, Mathematical Formula 26 is always established.

[0107] (4) In the case where I′<O and R′ are different signs from 1-b/a;

[0108] the values on the right side and the left side are negative, sothe larger R′/R is, the more difficult it is to establish MathematicalFormula 26.

[0109] In the above conditions (2) and (3), in the neighboring region,the higher the level of the input image is, the less compression isperformed, so there is no possibility that inversion of the gradientoccurs. Therefore, irrespective of the value of R′/R, MathematicalFormula 26 is always established. On the other hand, in the conditions(1) and (4), the larger the absolute value of R′/R is, the morepossibility there is that Mathematical Formula 26 is not satisfied.However, at least the ratio is the same as that of the input image, asdescribed above, Mathematical Formula 26 is established. In theembodiment, the threshold E of the epsilon filter 12 is variable, and asignal as similar as possible to the input image is outputted in theedge portion, that is, the threshold is controlled so that the largerthe edge strength G(x, y) is, the smaller the threshold E of the epsilonfilter 12 becomes, thereby the inversion of the gradient can be limitedto a minimum, and a natural image can be reproduced so as not togenerate the unnatural pattern 90 shown in FIG. 9B.

[0110] The level conversion function T(l) shown in FIG. 3 is just anexample, so any function can be used depending upon a purpose. Forexample, the function shown in Mathematical Formulas 1 or 2 may be used.

[0111] As described above, according to the embodiment, the edgestrength G(x, y) is calculated per position on the input image, and onthe basis of the edge strength G(x, y), the threshold E(x, y) of theepsilon filter 12 is controlled to filter the input image. Further, thecoefficient F(R(x, y)) calculated according to the output value R(x, y)from the epsilon filter 12 is multiplied by the input pixel value I(x,y) as shown in Mathematical Formula 16 to convert the pixel value perpixel and compress the dynamic range. Therefore, even if a plurality ofdifferent illuminations exist, the boundary thereof can be appropriatelyextracted, and an unnatural image pattern can be prevented from beinggenerated, thereby subjectively preferable compression of the dynamicrange can be achieved. In other words, by the use of the epsilon filter12, the illumination component is extracted from the input image, andthe illumination component is compressed, thereby while a local contrastis stored, the whole dynamic range is reduced, so a subjectivelypreferable reproduced image can be obtained. At this time, on the basisof the edge strength G(x, y), the threshold E of the epsilon filter 12is adaptively changed according to a local gradient of the pixel valueI(x, y), so the illumination boundary can be more accurately extractedthan in the case of using the linear lowpass filter or the fixedthreshold epsilon filter.

[0112] [Second Embodiment]

[0113] Next, a second embodiment of the invention will be describedbelow. In the description below, like components are denoted by likenumerals as of the first embodiment and will not be further explained.

[0114]FIG. 12 shows a structure of an image processing apparatusaccording to the second embodiment of the invention. Although a generalfunction of the image processing apparatus according to the embodimentis the same as that according to the first embodiment, it isdistinguished from the first embodiment by the fact that in theembodiment, instead of the divider 13 and the level conversion device 14(refer to FIG. 1), the image processing apparatus comprises acoefficient calculator 16 having functions of the divider 13 and thelevel conversion device 14. In other words, in the embodiment, thedynamic range is compressed on the basis of Mathematical Formula 16described in the first embodiment.

[0115] In the embodiment, the coefficient calculator 16 calculates acoefficient C(x, y) by applying the coefficient calculation functionF(l) shown in FIG. 13 to the output R(x, y) of the epsilon filter 12.The coefficient calculation function F(l) can be obtained by the use ofthe level conversion function T(l) by Mathematical Formula 15 asdescribed in the first embodiment. The coefficient C(x, y) calculated bythe coefficient calculator 16 is transmitted to the multiplier 15.

[0116] In the embodiment, a signal indicating the input image isdirectly inputted into the multiplier 15, and the coefficient C(x, y)calculated by the calculator 16 is inputted to the multiplier 15. In themultiplier 15, a multiplication of each pixel value I(x, y) of the inputimage by the corresponding coefficient C(x, y) is performed to reproducethe image signal, and as in the case of the first embodiment, the imagesignal O(x, y) which is the result of the multiplication, is outputtedto a transmission line, memory, a display apparatus or the like (all notshown).

[0117] Also in the embodiment, on the basis of the edge strength G(x,y), the threshold E(x, y) of the epsilon filter 12 is controlled tofilter the input image, and by multiplying the input pixel value I(x, y)by the coefficient F(R(x, y)) calculated according to the output valueR(x, y) from the epsilon filter 12, the pixel value is converted perpixel to compress the dynamic range, so the same effects as those of thefirst embodiment can be obtained.

[0118] [Third Embodiment]

[0119] Next, a third embodiment of the invention will be describedbelow. In the description below, like components are denoted by likenumerals as of the first embodiment and the second embodiment and willnot be further explained.

[0120]FIG. 14 shows a structure of an image processing apparatusaccording to the third embodiment of the invention. Although thestructure of the image processing apparatus according to the embodimentis substantially the same as that according to the first embodiment(refer to FIG. 1), it is distinguished from the first embodiment by thefact that in addition to the edge strength G(x, y) which is the outputfrom the edge strength calculator 10, the pixel value I(x, y) of theinput image is directly inputted into the threshold controller 11.

[0121] In the embodiment, a threshold controller 11A controls thethreshold E(x, y) used in the epsilon filter 12 in a subsequent stage bynot only the edge strength G(x, y) but also the pixel level of the inputimage. More specifically, the threshold controller 11A controls thethreshold E(x, y) so that the larger the pixel value I(x, y) of theinput image is, the larger the threshold E(x, y) becomes, and the largerthe edge strength G(x, y) is, the smaller the threshold E(x, y) becomes.

[0122] Such threshold control can be achieved as follows. For example,at first, by the use of a predetermined positive coefficient r (r 1.0),by Mathematical Formula 31, a temporary threshold Etmp(x, y) is set sothat the larger the pixel value I(x, y) of the input image is, thelarger the value of the temporary threshold Etmp(x, y) becomes.

[0123] After that, the threshold Etmp(x, y) is corrected by the edgestrength G(x, y) to determine the threshold E(x, y) actually used. Forexample, assuming that constants Emin and Emax for normalization inMathematical Formula 8 are 0.0 and 1.0, respectively, the coefficientG(x, y) corresponding to the edge strength is calculated by MathematicalFormula 8. The temporary threshold Etmp(x, y) is multiplied by thecoefficient G(x, y) as shown in Mathematical Formula 32 to determine afinal threshold E(x, y). Thereby, the threshold E(x, y) can becontrolled so that the larger the pixel value I(x, y) of the input imageis, the larger the value of the final threshold E(x, y) becomes, and thelarger the edge strength G(x, y) is, the smaller the value of the finalthreshold E(x, y) becomes.

[0124] The threshold E(x, y) of the epsilon filter 12 plays a role ofdiscriminating whether a spatial change in the input pixel occurs by achange in the illumination component or a change in a reflectivity of asurface of an object. Even in the case where the change in thereflectivity of the surface of the object is small, as long as theillumination level is large, a change in the pixel value I(x, y) becomeslarge. Therefore, how to discriminate between a large change in theillumination level and a small change in the reflectivity under strongillumination is an issue. In the embodiment, the threshold E(x, y) ofthe epsilon filter 12 is set so that the larger the pixel level is, thelarger the threshold E(x, y) becomes. Thereby, an influence of theillumination level on a change in the pixel value I(x, y) can bereduced, and the illumination component can be more appropriatelyextracted.

[0125] As described above, according to the embodiment, the thresholdE(x, y) of the epsilon filter 12 is controlled with consideration of notonly the edge strength G(x, y) but also the pixel level of the inputimage, so in the epsilon filter 12, the illumination component can bemore appropriately extracted.

[0126] [Fourth Embodiment]

[0127] Next, a fourth embodiment of the invention will be describedbelow. In the description below, like components are denoted by likenumerals as of the first, the second and the third embodiments and willnot be further explained.

[0128]FIG. 15 shows a structure of an image processing apparatusaccording to the fourth embodiment of the invention. Although thestructure of the image processing apparatus according to the embodimentis substantially the same as that according to the third embodiment(refer to FIG. 14), it is distinguished from the third embodiment by thefact that the threshold of the epsilon filter is controlled by the useof two kinds of thresholds Elo(x, y) and Eup(x, y) per pixel.

[0129] In the embodiment, a threshold controller 11B calculates the twokinds of thresholds Elo(x, y) and Eup(x, y) having different values fromeach other per pixel to control the threshold of an epsilon filter 12Aby the thresholds Elo(x, y) and Eup(x, y). In other words, in thethreshold controller 11B, for example, by the use of two predeterminedcoefficient with different values rl and ru (0.0 rl, ru 1.0), as shownin Mathematical Formulas 33A and 33B, two kinds of temporary thresholdsEtmplo(x, y) and Etmpup(x, y) are calculated. Then, the temporarythresholds Etmplo(x, y) and Etmpup(x, y) are outputted to the epsilonfilter 12A as a first threshold Elo(x, y) and a second threshold Eup(x,y), respectively.

[0130] In the embodiment, in the epsilon filter 12A, thresholdprocessing is performed with the two thresholds Elo(x, y) and Eup(x, y)calculated by the threshold controller 11B. More specifically,processing in the epsilon filter 12A is performed as below with, forexample, the structure shown in FIG. 16. The difference value D(x, y)calculated by the difference device 20 is transmitted to the absolutevalue calculator 21 as well as a sign discrimination device 24 in theembodiment.

[0131] In the sign discrimination device 24, the sign of the differencevalue D(x, y) is discriminated, and a discriminated result istransmitted to a switch 27.

[0132] A first comparator 25 selects a signal by Mathematical Formula 12by the use of the first threshold Elo(x, y) transmitted from thethreshold controller 11B as in the case of the first embodiment. Inother words, when a comparison between the value AD(dx, dy) calculatedby the absolute value calculator 21 and the threshold Elo(x, y) isperformed, according to the result of the comparison, either the valueI(x, y) of the pixel of interest or the value I(x+dx, y+dy) of theneighboring pixel is selected, and the selected value is outputted asthe value J(dx, dy).

[0133] In a second comparator 26, by the use of the second thresholdEup(x, y) transmitted from the threshold controller 11B, a signal isselected by Mathematical Formula 12 as in the case of the firstembodiment. In other words, when a comparison between the value AD(dx,dy) calculated by the absolute value calculator 21 and the thresholdEup(x, y) is performed, according to the result of the comparison,either the value I(x, y) of the pixel of interest or the value I(x+dx,y+dy) of the neighboring pixel is selected, and the selected value isoutputted as the value J(dx, dy).

[0134] In the switch 27, on the basis of the result of discrimination bythe sign discrimination device 24, either the output of the firstcomparator 25 or the second comparator 26 is selected to be transmittedto the linear lowpass filter 23. In the switch 27, for example, in thecase where the result of sign discrimination indicates positive, theoutput of the first comparator 25 is selected. On the other hand, in thecase where the result indicates negative, the output of the secondcomparator 26 is selected.

[0135] In the embodiment, when the value I(x+dx, y+dy) of theneighboring pixel is larger than the value (the value at the center ofthe neighboring region NB) I(x, y) of the pixel of current interest, thethreshold Eup(x, y) is used, and when it is smaller, the thresholdElo(x, y) is used. In other words, in the epsilon filter 12A, adifferent threshold depending on the high level side and the low levelside can be set. Specifically, when the second threshold Eup(x, y) islarger than the first threshold Elo(x, y), as in the case of the thirdembodiment, an influence of the illumination component included in achange in the pixel value can be reduced, and the illumination componentcan be more appropriately extracted.

[0136] [Fifth Embodiment]

[0137] Next, a fifth embodiment of the invention will be describedbelow. In the description below, like components are denoted by likenumerals as of the first, the second, the third and the fourthembodiments and will not be further explained.

[0138]FIG. 17 shows a structure of an image processing apparatusaccording to the fifth embodiment of the invention. Although thestructure of the image processing apparatus according to the embodimentis similar to that according to the first embodiment (refer to FIG. 1),it is distinguished from the first embodiment by the fact thatspecifically nonlinear transformation such as logarithmic transformationis performed on the pixel level of the input image.

[0139] In other words, in the embodiment, a logarithmic transformer 17is disposed in an input stage of a circuit, and at first, logarithmictransformation shown in Mathematical Formula 1 is performed on eachpixel value I(x, y) of the input image. Accordingly, the divider 13 forsubtracting the illumination component R(x, y) obtained by the epsilonfilter 12 and the multiplier 15 for multiplying the non-illuminationcomponent S(x, y) by the compressed illumination component CR(x, y) arereplaced with a subtracter 18 and an adder 19, respectively. It is basedupon a well-known fact that after the logarithmic transformation,multiplication and division become addition and subtraction,respectively.

[0140] A pixel value IL(x, y) of the input image which islogarithmically transformed by the logarithmic transformer 17 istransmitted to the edge strength calculator 10, the epsilon filter 12and the subtracter 18. In the edge strength calculator 10 and theepsilon filter 12, on the basis of the pixel value IL(x, y), the sameprocessing as that in the first embodiment is performed. On the otherhand, in the subtracter 18, a subtraction of the illumination componentR(x, y) obtained in the epsilon filter 12 from each pixel value IL(x, y)is performed to remove the illumination component from the input image,and the non-illumination component S(x, y) obtained by the result of thesubtraction is outputted to the adder 19. In the adder 19, thecorrection illumination component CR(x, y) is added to thenon-illumination component S(x, y) to reproduce the image signal, andthen as in the case of the first embodiment, the reproduced image signalO(x, y) is outputted to a transmission line, memory, a display apparatusor the like (all not shown).

[0141] The logarithmic transformation used in the embodiment has acompression effect of the dynamic range, so the higher the input levelis, the more the pixel value I(x, y) is compressed. Thereby, as in thecase of the third and the fourth embodiments, an influence of theillumination level on a spatial change in the pixel value can bereduced, and the illumination component can be more appropriatelyextracted.

[0142] In the embodiment, although an example that the logarithmictransformation is used as the nonlinear transformation is described, anynonlinear transformation other than the logarithmic transformation maybe used.

[0143] As described above, according to the image processing method orthe image processing apparatus of the invention, on the basis of theedge strength calculated per position on the input image, the thresholdused in the epsilon filter can be controlled, and according to theoutput value from the epsilon filter in which the threshold iscontrolled, the coefficient for converting the pixel value iscalculated, and the pixel value is converted per pixel by the use of thecalculated coefficient. Therefore, even if a plurality of differentilluminations exist, the boundary thereof can be appropriatelyextracted, and an unnatural image pattern can be prevented from beinggenerated, therefore subjectively preferable compression of the dynamicrange can be achieved.

[0144] Obviously many modifications and variations of the presentinvention are possible in the light of the above teachings. It istherefore to be understood that within the scope of the appended claimsthe invention may be practiced otherwise than as specifically described.

[0145] (MATHEMATICAL FORMULA 1) $\begin{matrix}{{T(l)} = {\left( \frac{l}{Lmax} \right)^{g} \times {Lmax}}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 1} \right)\end{matrix}$

[0146] (MATHEMATICAL FORMULA 2) $\begin{matrix}{{T(l)} = {\frac{\log (l)}{\log ({Lmax})} \times {Lmax}}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 2} \right)\end{matrix}$

[0147] (MATHEMATICAL FORMULA 3) $\begin{matrix}{{C(l)} = {\sum\limits_{k = 0}^{l}{H(k)}}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 3} \right)\end{matrix}$

[0148] (MATHEMATICAL FORMULA 4) $\begin{matrix}{{T(l)} = {\frac{C(l)}{Fmax} \times {Lmax}}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 4} \right)\end{matrix}$

[0149] (MATHEMATICAL FORMULA 5)

O(x, y)=log(I(x, y))−log(LPF(I(x, y)))

[0150] (MATHEMATICAL FORMULA 6)

G(x, y)=/I(x−d, y)−I(x+d, y)/ +/I(x, y−d)−I(x, y+d)/

[0151] (MATHEMATICAL FORMULA 7) $\begin{matrix}\begin{matrix}{{G\left( {x,y} \right)} = {{\sum\limits_{{dy} = {- d}}^{d}\left\lbrack {{I\left( {{x - d},{y + {dy}}} \right)} -} \right.}}} \\{{\left. {I\left( {{x + d},{y + {dy}}} \right)} \right\rbrack } +} \\{{{\sum\limits_{{dy} = {- d}}^{d}\left\lbrack {{I\left( {{x + {dx}},{y - d}} \right)} -} \right.}}} \\{\left. {I\left( {{x + {dx}},{y + d}} \right)} \right\rbrack }\end{matrix} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 7} \right)\end{matrix}$

[0152] (MATHEMATICAL FORMULA 8) $\begin{matrix}{{E\left( {x,y} \right)} = \left\{ \begin{matrix}{Emin} & {{\cdots \quad {G\left( {x,y} \right)}} > {Gmax}} \\{{\frac{{Gmax} - {G\left( {x,y} \right)}}{{Gmax} - {Gmin}}\left( {{Emax} - {Emin}} \right)} + {Emin}} & {{\cdots \quad {Gmin}} \leqq {G\left( {x,y} \right)} \leqq {Gmax}} \\{Emax} & {{\cdots \quad {G\left( {x,y} \right)}} < {Gmin}}\end{matrix} \right.} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 8} \right)\end{matrix}$

[0153] (MATHEMATICAL FORMULA 9) $\begin{matrix}{{S\left( {x,y} \right)} = \frac{I\left( {x,y} \right)}{R\left( {x,y} \right)}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 9} \right)\end{matrix}$

[0154] MATHEMATICAL FORMULA 10)

CR(x, y)=T(R(x, y))

[0155] (MATHEMATICAL FORMULA 11)

O(x, y)=S(x, y)CR(x, y)

[0156] $\begin{matrix}{{J\left( {{dx},{dy}} \right)} = \left\{ \begin{matrix}{I\left( {x,y} \right)} & {{\cdots \quad {{AD}\left( {{dx},{dy}} \right)}} > {E\left( {x,y} \right)}} \\{I\left( {{x + {dx}},{y + {dy}}} \right)} & {{\cdots \quad {{AD}\left( {{dx},{dy}} \right)}} \leqq {E\left( {x,y} \right)}}\end{matrix} \right.} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 12} \right)\end{matrix}$

$\begin{matrix}{{R\left( {x,y} \right)} = {\sum\limits_{{({{dx},{dy}})} \in {NB}}{{a\left( {{dx},{dy}} \right)}{J\left( {{dx},{dy}} \right)}}}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 13} \right)\end{matrix}$

$\begin{matrix}{{R\left( {x,y} \right)} = {\frac{1}{N}{\sum\limits_{{({{dx},{dy}})} \in {NB}}{J\left( {{dx},{dy}} \right)}}}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 14} \right)\end{matrix}$

$\begin{matrix}{{F(l)} = \frac{T(l)}{l}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 15} \right)\end{matrix}$

[0157] (MATHEMATICAL FORMULA 16)

O(x, y)=I(x, y)F(R(x, y))

[0158] $\begin{matrix}{{C\quad \min} = \frac{{CR}\quad \max}{R\quad \max}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 17} \right)\end{matrix}$

[0159] (MATHEMATICAL FORMULA 18)

T(l)=al+b . . . l≧lk

[0160] $\begin{matrix}{{c5} = {\frac{{a \times {l5}} + b}{l5} = {a + \frac{b}{l5}}}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 19} \right)\end{matrix}$

$\begin{matrix}{N < \frac{2E}{a}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 20} \right)\end{matrix}$

[0161] (MATHEMATICAL FORMULA 21) sign(O′(x, y))=sign(I′(x, y))$\begin{matrix}{{O\left( {x,y} \right)} = {{I\left( {x,y} \right)}\frac{{CR}\left( {x,y} \right)}{R\left( {x,y} \right)}}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 22} \right)\end{matrix}$

$\begin{matrix}\left\{ \begin{matrix}{{\frac{R^{\prime}\left( {x,y} \right)}{R\left( {x,y} \right)} - \frac{{CR}^{\prime}\left( {x,y} \right)}{{CR}\left( {x,y} \right)}} \leqq \frac{I^{\prime}}{I}} & \cdots & {I^{\prime} \geqq 0} \\{{\frac{R^{\prime}\left( {x,y} \right)}{R\left( {x,y} \right)} - \frac{{CR}^{\prime}\left( {x,y} \right)}{{CR}\left( {x,y} \right)}} > \frac{I^{\prime}}{I}} & \cdots & {I^{\prime} < 0}\end{matrix} \right. & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 23} \right)\end{matrix}$

[0162] MATHEMATICAL FORMULA 24)

CR=aR

[0163] MATHEMATICAL FORMULA 25)

CR)′=bR′

[0164] $\begin{matrix}\left\{ \begin{matrix}{{\frac{R^{\prime}}{R}\left( {1 - \frac{b}{a}} \right)} \leqq \frac{I^{\prime}}{I}} & \cdots & {I^{\prime} \geqq 0} \\{{\frac{R^{\prime}}{R}\left( {1 - \frac{b}{a}} \right)} > \frac{I^{\prime}}{I}} & \cdots & {I^{\prime} < 0}\end{matrix} \right. & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 26} \right)\end{matrix}$

$\begin{matrix}{{1 - \frac{b}{a}} \leqq 1} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 27} \right)\end{matrix}$

$\left. \begin{matrix}\begin{matrix}\begin{matrix}\left( {{MATHEMATICAL}\quad {FORMULA}\quad 28A} \right) \\{{{\frac{I^{\prime}}{I}} \geqq {\frac{R^{\prime}}{R}}}\quad}\end{matrix} \\\left( {{MATHEMATICAL}\quad {FORMULA}\quad 28B} \right)\end{matrix} \\{{{sign}\left( {I^{\prime}\left( {x,y} \right)} \right)} = {{sign}\left( {R^{\prime}\left( {x,y} \right)} \right)}}\end{matrix} \right\}$

$\begin{matrix}{{\frac{R^{\prime}}{R}\left( {1 - \frac{b}{a}} \right)} \leqq {\frac{I^{\prime}}{I}\left( {1 - \frac{b}{a}} \right)} \leqq \frac{I^{\prime}}{I}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 29} \right)\end{matrix}$

[0165] (MATHEMATICAL FORMULA 30) $\begin{matrix}{\frac{R^{\prime}\left( {x,y} \right)}{R\left( {x,y} \right)} = \frac{I^{\prime}\left( {x,y} \right)}{I\left( {x,y} \right)}} & \left( {{MATHEMATICAL}\quad {FORMULA}\quad 30} \right)\end{matrix}$

[0166] (MATHEMATICAL FORMULA 31)

Etmp(x, y)=rI(x, y)

[0167] (MATHEMATICAL FORMULA 32)

E(x, y)=Etmp(x, y)G(x, y)

[0168] (MATHEMATICAL FORMULA 33A)

Etmplo(x, y)=rl×I(x, y)

[0169] (MATHEMATICAL FORMULA 33B)

Etmpup(x, y)=ru×I(x, y)

1. An image processing method for converting an inputted image into animage with a relatively smaller dynamic range, the method comprising thesteps of: calculating an edge strength per position on an input image;controlling a threshold of an epsilon filter on the basis of thecalculated edge strength; filtering the input image by the epsilonfilter by use of the threshold controlled in the step of controlling thethreshold; and calculating a coefficient for converting a pixel valueaccording to an output value in the step of filtering, and convertingthe pixel value per pixel by the calculated coefficient.
 2. An imageprocessing method according to claim 1, wherein in the step ofcalculating the edge strength, the larger a primary differential valueof a pixel value in a neighboring region of a pixel of interest is, thelarger a value is calculated as the edge strength.
 3. An imageprocessing method according to claim 1, wherein in the step ofcontrolling the threshold, the threshold is controlled so that thelarger the edge strength is, the smaller the threshold becomes.
 4. Animage processing method according to claim 1, wherein in the step ofcontrolling the threshold, the threshold is controlled so that thelarger the pixel value of the input image is, the larger the thresholdbecomes, and the larger the edge strength is, the smaller the thresholdbecomes.
 5. An image processing method according to claim 1, wherein inthe step of controlling the threshold, two thresholds with differentvalues are calculated, and in the step of filtering, a differentthreshold is used depending upon whether a value of a neighboring pixelis smaller or larger than a value of a pixel of interest.
 6. An imageprocessing method according to claim 5, wherein in the step offiltering, in the case where the value of the neighboring pixel islarger than the value of the pixel of interest, the threshold with alarger value out of the two thresholds is used, and in the case wherethe value of the neighboring pixel is smaller than the value of thepixel of interest, the threshold with a smaller value is used.
 7. Animage processing method according to claim 1, further comprising thestep of: performing nonlinear transformation on a pixel level of theinput image, wherein the input image after the nonlinear transformationis filtered by the epsilon filter.
 8. An image processing methodaccording to claim 7, wherein the nonlinear transformation islogarithmic transformation.
 9. An image processing apparatus forconverting an inputted image into a image with a relatively smallerdynamic range, the image processing apparatus comprising: an edgestrength calculating means for calculating an edge strength per positionon an input image; an epsilon filter for filtering the input image byuse of a predetermined threshold; a threshold control means forcontrolling the threshold used in the epsilon filter on the basis of theedge strength calculated by the edge strength calculating means; and apixel value conversion means for calculating a coefficient forconverting a pixel value according to an output value from the epsilonfilter and converting the pixel value per pixel by the calculatedcoefficient.
 10. An image processing apparatus according to claim 9,wherein the edge strength calculating means is configured so that thelarger a primary differential value of a pixel value in a neighboringregion of a pixel of interest is, the larger a value is calculated asthe edge strength.
 11. An image processing apparatus according to claim9, wherein in the threshold control means, the threshold is controlledso that the larger the edge strength is, the smaller the thresholdbecomes.
 12. An image processing apparatus according to claim 9, whereinin the threshold control means, the threshold is controlled so that thelarger the pixel value of the input image is, the larger the thresholdbecomes, and the larger the edge strength is, the smaller the thresholdbecomes.
 13. An image processing apparatus according to claim 9, whereinthe threshold control means is configured to calculate two thresholdswith different values, and use a different threshold depending uponwhether a value of a neighboring pixel is smaller or larger than a pixelof interest.
 14. An image processing apparatus according to claim 13,wherein the epsilon filter is configured so that in the case where thevalue of the neighboring pixel is larger than the value of the pixel ofinterest, the threshold with a larger value out of the two thresholds isused, and in the case where the value of the neighboring pixel issmaller than the value of the pixel of interest, the threshold with asmaller value is used.
 15. An image processing apparatus according toclaim 9, further comprising: a nonlinear transformation means forperforming nonlinear transformation on a pixel level of the input image,wherein the epsilon filter is configured to filter the input image afterthe nonlinear transformation.
 16. An image processing apparatusaccording to claim 15, wherein the nonlinear transformation meansperforms logarithmic transformation on the pixel level of the inputimage.